Adaptive control of stochastic systems with unknown disturbance distribution: discounted criteria

Abstract We consider a class of discrete-time stochastic control systems, with Borel state and action spaces, and possibly unbounded costs. The processes evolve according to the equation xt+1 = F (xt, at, ξt), t = 0, 1, . . . , where the ξt are i.i.d. random vectors whose common distribution is unknown. Assuming observability of {ξt}, we use the empirical estimator of its distribution to construct adaptive policies which are asymptotically discounted cost optimal . AMS 1991 subject classifications: 93E20, 90C40.